Module 4 - Classification

Overview

During this week, we will focus on methods that can be used to solve classification problems. We will start by talking about the basis for decision rules in classification, which connects machine learning to statistics via likelihood ratio tests and loss functions. We will introduce the most basic classification model, which is called logistic regression, and which is in many ways the analog of linear regression for classification problems. We will go over a number of important instances of logistic regression and then discuss the intricacies of evaluating classification models.

Learning Objectives

• Classification with Full Data: Likelihood Ratio Tests, Neymon-Pearson Lemma, and Loss-Functions
• Logistic Regression: Multiple and Multinomial
• Classification Metrics

Readings

• ISLP (Introduction to Statistical Learning): Sections 4.1-4.3

Additional Reading:

If you want to go deeper into this topic:

• Hands on Machine Learning Chapter 3. This is a very practical take on classification
• Patterns, Predictions, and Actions Chapter 2 Very good exposition of the theory

Videos

https://www.youtube.com/watch?v=ju3J7iRy6xI&list=PLoROMvodv4rOzrYsAxzQyHb8n_RWNuS1e&index=15

• Section 4.1: Introduction to Classification
• Section 4.2: Logistic Regression
• Section 4.3: Multivariate Logistic Regression
• Section 4.4: Logistic Regression Case Control Sampling and Multi-Class

Coding Videos

• Section 4.1: Simple Linear Regression